Skip to content

feat: add LiteLLM as AI gateway provider#203

Open
RheagalFire wants to merge 1 commit into
siddsachar:mainfrom
RheagalFire:feat/add-litellm-provider
Open

feat: add LiteLLM as AI gateway provider#203
RheagalFire wants to merge 1 commit into
siddsachar:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire

Copy link
Copy Markdown

Summary

Add LiteLLM as a built-in provider, giving users access to 100+ LLM providers (Anthropic, Azure, Bedrock, Vertex, Groq, Ollama, etc.) through a single OpenAI-compatible proxy.

LiteLLM proxy runs locally and translates requests to any supported provider's native format. Since it speaks OpenAI protocol, it reuses the existing OPENAI_CHAT transport with zero new transport code.

Changes:

  • src/row_bot/providers/catalog.py - added litellm ProviderDefinition (base_url: http://localhost:4000/v1, transport: OPENAI_CHAT)
  • src/row_bot/providers/runtime.py - added litellm to configured provider list
  • tests/test_litellm_provider.py - 3 unit tests

Type of change

  • Bug fix
  • Feature
  • Docs
  • Refactor
  • Test-only change
  • Build / CI / release tooling

Risk area

  • Agent / prompts / tools
  • Designer
  • Memory / knowledge graph
  • Channels / external integrations
  • Installers / auto-update
  • UI only
  • Other

Testing

  • I ran python tests/test_suite.py
  • I added or updated tests
  • I manually tested the affected user flow
  • Not applicable, docs-only change

python tests/test_suite.py: 1143 pass, 103 fail (all pre-existing failures from missing optional deps nicegui and langgraph.checkpoint.sqlite, unrelated to this change).

tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_registered PASSED
tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_definition PASSED
tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_uses_openai_compatible_transport PASSED
3 passed in 0.26s

Existing provider catalog tests: 41 passed (1 pre-existing nicegui failure).

Release notes

  • User-visible change, release notes needed

Users can now select LiteLLM as a provider and point it at their LiteLLM proxy (http://localhost:4000) to access any of 100+ supported LLM providers.

Checklist

  • Branch is based on latest main
  • No direct secrets, API keys, local paths, or private data included
  • The change is focused and does not include unrelated cleanup
  • Windows/macOS behavior considered where relevant

@RheagalFire RheagalFire requested a review from siddsachar as a code owner June 15, 2026 18:47
@RheagalFire

Copy link
Copy Markdown
Author

@siddsachar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant